Surgical system with voice control

ABSTRACT

A surgical system includes a plurality of voice sensors located in a surgical environment and configured to detect sound and generate a first plurality of signals. The surgical system also includes a position indicator, in proximity to a designated user, configured to indicate a first position of the designated user and generate a second signal representative of the first position. The surgical system further includes a processor configured to receive the first plurality of signals and the second signal and determine, based on the first plurality of signals, a second position. The processor is also configured to compare the detected sound with registered voice command of the designated user stored in a memory to verify the designated user&#39;s credentials, and send a command signal to a surgical instrument to carry out an operation related to the voice command based on at least one of the verification of the designated user&#39;s credentials, the first position and the second position.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/887,014 entitled “Surgical System With Voice Control” filed May 29,2020, which is a continuation of U.S. patent application Ser. No.16/013,343, entitled “Surgical System With Voice Control,” filed Jun.20, 2018 (now U.S. Pat. No. 10,667,878), which is a continuation of U.S.patent application Ser. No. 15/383,564, entitled “Surgical System WithVoice Control” filed Dec. 19, 2016, (now U.S. Pat. No. 10,028,794),which are incorporated herein by reference in their entireties.

FIELD

Methods and devices are provided for minimally invasive surgery, and inparticular for providing voice control of a surgical instrument.

BACKGROUND

Minimally invasive surgical (MIS) instruments are often preferred overtraditional open surgical instruments due to the reduced post-operativerecovery time and minimal scarring. Laparoscopic surgery is one type ofMIS procedure in which one or more small incisions are formed in theabdomen and a trocar is inserted through the incision to form a pathwaythat provides access to the abdominal cavity. The trocar is used tointroduce various instruments and tools into the abdominal cavity, aswell as to provide insufflation to elevate the abdominal wall above theorgans. The instruments and tools can be used to engage and/or treattissue in a number of ways to achieve a diagnostic or therapeuticeffect. Endoscopic surgery is another type of MIS procedure in whichelongate flexible shafts are introduced into the body through a naturalorifice.

Although traditional minimally invasive surgical instruments andtechniques have proven highly effective, newer systems may provide evenfurther advantages. For example, traditional minimally invasive surgicalinstruments often deny the surgeon the flexibility of tool placementfound in open surgery. Difficulty is experienced in approaching thesurgical environment with the instruments through the small incisions.Additionally, the added length of typical endoscopic instruments oftenreduces the surgeon's ability to feel forces exerted by tissues andorgans on the end effector. Furthermore, coordination of the movement ofthe end effector of the instrument as viewed in the image on thetelevision monitor with actual end effector movement is particularlydifficult, since the movement as perceived in the image normally doesnot correspond intuitively with the actual end effector movement.Accordingly, lack of intuitive response to surgical instrument movementinput is often experienced. Such a lack of intuitiveness, dexterity, andsensitivity of endoscopic tools has been found to be an impediment inthe increased the use of minimally invasive surgery.

Over the years a variety of minimally invasive robotic systems have beendeveloped to increase surgical dexterity as well as to permit a surgeonto operate on a patient in an intuitive manner. Telesurgery is a generalterm for surgical operations using systems where the surgeon uses someform of remote control, e.g., a servomechanism, or the like, tomanipulate surgical instrument movements, rather than directly holdingand moving the tools by hand. In such a telesurgery system, the surgeonis typically provided with an image of the surgical environment on avisual display at a location remote from the patient. The surgeon cantypically perform the surgical procedure at the location remote from thepatient whilst viewing the end effector movement on the visual displayduring the surgical procedure. While viewing typically athree-dimensional image of the surgical environment on the visualdisplay, the surgeon performs the surgical procedures on the patient bymanipulating master control devices at the remote location, which mastercontrol devices control motion of the remotely controlled instruments.

While significant advances have been made in the field of minimallyinvasive surgery, there remains a need for improved methods, systems,and devices for providing seamless control of surgical instrument basedon voice commands.

SUMMARY

Methods, devices, and systems are provided for the use of voice commandsto control one or more surgical instruments. In addition to issuing acommand signal to a surgical instrument, a processor associated with thesystem also authenticates voice commands to ensure that the voicecommands are actionable commands. A command is actionable when it hasbeen authenticated by at least one authentication technique. In oneembodiment, more than one authentication technique is required before avoice command is regarded as actionable.

A surgical system includes a plurality of voice sensors located in asurgical environment and configured to detect sound and generate a firstplurality of signals. The surgical system also includes a positionindicator, in proximity to a designated user, configured to indicate afirst position of the designated user and to generate a second signalrepresentative of the first position. The surgical system furtherincludes a processor configured to receive the first plurality ofsignals and the second signal and determine, based on the firstplurality of signals, a second position. The processor is alsoconfigured to compare the detected sound with a registered voice commandof the designated user stored in a memory to verify the designateduser's credentials, and to send a command signal to a surgicalinstrument to carry out an operation related to the voice command basedon at least one of the verification of the designated user'scredentials, the first position, and the second position.

In one embodiment the detected sound is a voice command generated by thedesignated user. The voice command is detected by at least one voicesensor in a headset worn by the designated user. In one implementationthe headset includes a camera configured to detect a motion of lips ofthe designated user during the generation of the voice command and tosend a signal related to the detection to the processor. In anotherimplementation the processor is configured to determine whether thedetected motion of lips of the designated user matches the generatedvoice commands and to send the command signal when the detected motionof lips of the designated user matches the generated voice commands.

In one embodiment the voice command is detected by at least one voicesensor disposed on the surgical instrument. In another embodiment thesecond position is determined based on one or more delays in detectionof the sound by one or more of the plurality of voice sensors. In yetanother embodiment, the processor is configured to send the commandsignal when the difference between the first position and the secondposition is less than a predetermined value.

In one embodiment, the processor is configured to compare the detectedsound with the registered voice command using a voice recognitionalgorithm. In another embodiment, the surgical instrument includes aswitch and the surgical instrument carries out the operation related tothe voice command when the switch is actuated. In yet anotherembodiment, the processor is configured to send the command signal basedon at least two of the verification of the designated user'scredentials, the first position, and the second position.

In another aspect, a robotic surgical system includes a robotic arm, atool assembly removably coupled to the robotic arm and comprising ashaft extending distally from a housing and an end effector coupled to adistal end of the shaft, the end effector being configured to treattissue. The robotic surgical system also includes a plurality of voicesensors located in a surgical environment housing the robotic arm andconfigured to detect sound and generate a first plurality of signals.The robotic surgical system further includes a position indicator, inproximity to a designated user, configured to indicate a first positionof the designated user and generate a second signal representative ofthe first position. The robotic surgical system also includes aprocessor configured to receive the first plurality of signals and thesecond signal, and determine, based on the first plurality of signals, asecond position. The processor is also configured to compare thedetected sound with a registered voice command of the designated userstored in a memory to verify the designated user's credentials, and tosend a command signal to the tool assembly to carry out an operationrelated to the voice command based on at least one of the verificationof the designated user's credentials, the first position and the secondposition.

In one embodiment, the second position is determined based on one ormore delays in detection of the sound by one or more of the plurality ofvoice sensors.

In another aspect a method comprises detecting, by a plurality of voicesensors, sound produced in a surgical environment; indicating, by aposition sensor, a first position associated with a designated user;determining, based on the detected sound, a second position; comparingthe detected sound with registered voice commands stored in a memory toverify the designated user's credentials; and sending a command signalto a surgical instrument to carry out an operation related to the voicecommand based on at least one of the verification of the user'scredential, the first position, and the second position. In one aspect,the, designated user's voice is preregistered for command authenticityand the stored in memory.

In one implementation of the method, the second position is determinedbased on one or more delays in detection of the sound by one or more ofthe plurality of voice sensors. In another aspect, the processor sendsthe command signal when the difference between the first position andthe second position is less than a predetermined value. In yet anotherimplementation of the method, the processor sends the command signalbased on at least two of the verification of the designated user'scredentials, the first position, and the second position.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be more fully understood from the following detaileddescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 illustrates a perspective view of an embodiment of a surgicalrobotic system that includes a patient-side portion and a user-sideportion;

FIG. 2 illustrates a surgical environment comprising a surgical systemconfigured to allow voice control of a surgical instrument;

FIG. 3 illustrates an example of the surgical instrument;

FIG. 4 illustrates an example of a head set worn by a user of thesurgical system;

FIG. 5 is a schematic illustration of a computer system configured togenerate a plurality of command signals for use with the control systemdescribed herein.

DETAILED DESCRIPTION

Certain exemplary embodiments will now be described to provide anoverall understanding of the principles of the structure, function,manufacture, and use of the devices and methods disclosed herein. One ormore examples of these embodiments are illustrated in the accompanyingdrawings. Those skilled in the art will understand that the devices andmethods specifically described herein and illustrated in theaccompanying drawings are non-limiting exemplary embodiments and thatthe scope of the present invention is defined solely by the claims. Thefeatures illustrated or described in connection with one exemplaryembodiment may be combined with the features of other embodiments. Suchmodifications and variations are intended to be included within thescope of the present invention.

Further, in the present disclosure, like-named components of theembodiments generally have similar features, and thus within aparticular embodiment each feature of each like-named component is notnecessarily fully elaborated upon. Additionally, to the extent thatlinear or circular dimensions are used in the description of thedisclosed systems, devices, and methods, such dimensions are notintended to limit the types of shapes that can be used in conjunctionwith such systems, devices, and methods. A person skilled in the artwill recognize that an equivalent to such linear and circular dimensionscan easily be determined for any geometric shape. Sizes and shapes ofthe systems and devices, and the components thereof, can depend at leaston the anatomy of the subject in which the systems and devices will beused, the size and shape of components with which the systems anddevices will be used, and the methods and procedures in which thesystems and devices will be used.

The systems, devices, and methods disclosed herein can be implementedusing a robotic surgical system. WIPO Patent Publication No. WO2014/151621 filed on Mar. 13, 2014 and entitled “Hyperdexterous SurgicalSystem” is incorporated by reference.

In general, a surgical system is described that allows a user (e.g., asurgeon) in a surgical environment to control the operation of asurgical instrument by voice commands. The surgical instrument can bepart of a robotic surgical system. A surgeon performing a surgery (e.g.,minimally invasive surgery) may have to perform several surgical tasks,which occur simultaneously and/or which may be actuated by a roboticsurgical system. Therefore, it can be desirable for the surgeon to beable control the operation of a surgical instrument by using voicecommands.

A surgical system that allows for voice control of surgical instrumentmust be configured in such a way that the system first ensures that thecommand is an actionable command that is made by an authorized user(e.g., a surgeon). Thus, the system must authenticate the voice commandgenerated by the user. The authentication protocol can be a single stepauthentication protocol or a multi-step authentication protocol. In oneaspect, the system will register a voice command for a designated user,which can be stored in memory, and used for subsequent authentication ofvoice commands. The authentication protocol involves one or more of thefollowing: verifying that the voice command is issued by a designated orauthorized user (such as by comparison with a registered voice),verifying the authenticity of the voice command based on the location ofthe source of the voice command, and verifying the authenticity of voicecommand by capturing the motion of the lips of the designated user anddeciphering the corresponding spoken phrase. Optionally, the spokenphrase captured by detecting the motion of the designated user's lipscan be compared to the detected voice command.

In one example, the surgical system can use a voice recognitiontechnology to ensure that the voice command is generated by a designateduser. Additionally or alternately, the surgical system can determine theauthenticity of the voice command based on the location of the source ofthe voice command. For example, the surgical system can detect theposition of the designated user by a position indicator, and compare thedetected position with a location of a sound source in the surgicalenvironment measured using multiple voice sensors. Based on thecomparison, the surgical system can determine whether the designateduser is also the source of sound in the surgical area. The surgicalsystem can also verify the authenticity of a voice command by capturingthe motion of the lips of the designated user and deciphering thecorresponding spoken phrase. Optionally, the deciphered phrase can becompared with the detected voice command. Following authentication ofthe voice command based on one or more of the techniques mentionedabove, the surgical system will generate a command signal to control theoperation of the surgical instrument in a manner consistent with thevoice command.

FIG. 1 is a perspective view of one embodiment of a surgical roboticsystem 100 that can be used in telesurgery. The system 100 includes apatient-side portion 110 that is positioned adjacent to a patient 112,and a user-side portion 111 that is located a distance from the patient,either in the same room and/or in a remote location. The patient-sideportion 110 generally includes one or more robotic arms 120 and one ormore tool assemblies 130 that are configured to releasably couple to arobotic arm 120. The user-side portion 111 generally includes a visionsystem 113 for viewing the patient 112 and/or surgical environment, anda control system 115 for controlling the movement of the robotic arms120 and each tool assembly 130 during a surgical procedure.

The control system 115 can have a variety of configurations and it canbe located adjacent to the patient, e.g., in the operating room, remotefrom the patient, e.g., in a separate control room, or it can bedistributed at two or more locations. As an example of a dedicatedsystem, a dedicated system control console can be located in theoperating room, and a separate console can be located at a remotelocation. The control system 115 can include components that enable auser to view a surgical environment of a patient 112 being operated onby the patient-side portion 110 and/or to control one or more parts ofthe patient-side portion 110 (e.g., to perform a surgical procedure atthe surgical environment). In some embodiments, the control system 115can also include one or more manually-operated input devices, such as ajoystick, exoskeletal glove, a powered and gravity-compensatedmanipulator, or the like. These input devices can control teleoperatedmotors which, in turn, control the movement of the surgical system,including the robotic arms 120 and tool assemblies 130.

The patient-side portion can also have a variety of configurations. Asdepicted in FIG. 1 , the patient-side portion 110 can couple to anoperating table 114. However, in other embodiments, the patient-sideportion 110 can be mounted to a wall, to the ceiling, to the floor, orto other operating room equipment. Further, while the patient-sideportion 110 is shown as including two robotic arms 120, more or fewerrobotic arms 120 can be included. Furthermore, the patient-side portion110 can include separate robotic arms 120 mounted in various positions,such as relative to the operating table 114. Alternatively, thepatient-side portion 110 can include a single assembly that includes oneor more robotic arms 120 extending therefrom.

FIG. 2 is a schematic view of a surgical environment 200 that includes asurgical system configured to allow voice control of a surgicalinstrument 216. The surgical environment can include several personnel(e.g., surgeons or other medical professionals) 202, 204, 206. A surgeon(e.g., the person designated with reference numeral 204) can be assignedas a designated user to control the operation of the surgical instrument216. A position indicator 210 is positioned in proximity to designateduser 204 and it is configured to detect the position of the designateduser 204. For example, the position indicator 210 can be secured to thebody of the designated user (e.g., located in a wrist band or a headsetworn by the designated user, or placed on the chest of the designateduser). The surgical system also includes several voice sensors (e.g.,212, 214, etc.) configured to detect sound produced in the surgicalenvironment 200 (e.g., voice of the personnel 202, 204, 206). Thesurgical system further includes a processor 250 that is responsible forvoice command authentication as well as the control of the operation ofthe surgical instrument 216. For example, the processor 250 cancommunicate with the voice sensors 212, 214 positon indicator 210, andsurgical instrument 216. Further, processor 250 can save data to and/orretrieve data from a database 252. Signals in the surgical system (e.g.,between processor 250 and the position indicator 210, voice sensors 212and 214, and processor 250, surgical instrument 216 etc.) can becommunicated wirelessly (e.g., via Bluetooth, WiFi, etc.) or through adata cable (e.g., optical fiber, coaxial cable, etc.). It is understoodthat the processor and the surgical system can be part of a roboticsurgical system, however the present disclosure is applicable tonon-robotic surgical systems as well.

As noted above, the surgical system can allow a user (e.g., designateduser 204) to control the surgical instrument 216 once the systemsuccessfully completes one or more authentication steps. A user can beassigned as a designated user prior to the start of the surgicalprocedure. For example, a user can specify to the surgical system thathe/she is the designated user by verifying his/her credentials (e.g., bylogging into the system using a password, by swiping an identification(ID) card, etc.). The surgical system can ensure that informationassociated with the designated user that is necessary to allow thedesignated user to use voice control to actuate the surgical instrument216 (e.g., the user's voice) is stored in the database 252. Suchinformation can include, for example, voice recordings of the designateduser. The designated user 204 (or any other personnel) can also indicateto the surgical system the identity of the surgical instrument 216 to beoperated via voice control. In one aspect the designated user 204 can beassigned a positon indicator 210 that allows the system to locate thedesignated user in the surgical environment. The position indicator 210can wirelessly (e.g., Bluetooth, WiFi, etc.) communicate with theprocessor 250, either directly or through another device.

The surgical system can also include voice sensors (e.g., 212, 214,etc.) that can be distributed in the surgical environment 200. Forexample, the voice sensors can be located on walls in the surgicalenvironment, on the surgical instrument 216, on the operating table,mounted on stands affixed to the floor, etc. The voice sensors candetect a sound generated in the surgical environment and send a signalto the processor 250 indicating the time at which the sound wasdetected. The processor 250 receives signals indicating the time ofdetection from the various voice sensors in the surgical environment200, and the location of the source of sound can be ascertained using atriangulation algorithm. In one embodiment, the triangulation algorithmcan be based on the assumption that the time at which a voice sensordetects the sound is proportional to the distance between the source ofsound and the voice sensor. Voice sensors that are farther away from thesource of sound detect the sound at a later time. For example, sensor212 (or sensor 214) will detect the sound produced by designated user204 after a time period that is proportion of the distance “a” 222 (or“b” 224). The processor 250 can determine the difference in time ofdetection of the various sensors. The processor 250 can use thisinformation, information related to the position of the various voicesensors/separation between the sensors (e.g., separation of “d” 220between sensors 212 and 214), and the speed of sound in air in thetriangulation algorithm (e.g., algorithm that solves an “inverseproblem”) to determine the position of the source of sound.

If the position of the designated user 204 indicated by the positionindicator 210 is within a predetermined distance (e.g., 1 foot, 2 feet,3 feet, etc., which can be stored in database 252) from the determinedlocation of the source of sound, the processor 250 can conclude that thedesignated user 204 is also the source of the detected sound. In thiscase, the processor 250 will generate a command signal for the surgicalinstrument that is consistent with the voice command issued by thedesignated user 204. In the event that the sound generated by a person(e.g., 202, 206) who is not the designated user 204 is determined not tobe located within the predetermined distance, based on the positionmeasurement from the position indicator 210 and the determined locationof the source of sound, the processor 250 will not generate a commandsignal for the surgical instrument 216.

As noted above, one or more voice sensors can be located in proximity tothe designated user to detect sound generated by the user. For example,a microphone can be attached to a headset worn by the designated user,or it can be hung around the designated user's neck. The microphone canbe located, for example, a specified distance (e.g., 6 -12 inches) fromthe user's mouth. A variety of microphones can be used, but in oneembodiment the microphone has a limited range of sound detection. Forexample, the microphone may be configured to detect sound generatedwithin a 6-12 inch radius. Such a sound pickup limitation can increasethe possibility that the microphone captures the sound generated only bythe designated user.

Once the microphone captures designated user's voice, a signal relatedto the voice is communicated to the processor 250. The processor 250then compares the designated user's voice with the voice recordings ofthe designated user stored in the database 252. In one embodiment avoice recognition algorithm can be used to verify a match between thestored sample of the designated user's voice and the voice detected bythe voice sensor(s). Additionally, the captured designated user's voicecan be stored in the database 252 and can be used to train the surgicalsystem (e.g., using machine learning algorithms) to detect voice commandof the designated user with greater accuracy. The processor 250 can alsodetermine whether the designated user's voice matches predeterminedvoice commands for the surgical instrument 216. For example, theprocessor 250 can retrieve a voice command data file associated with thesurgical instrument 216 from the database 252, and compare them with thedesignated user's voice. For example, the voice commands for anendocutter can include “move right,” “move left,” “move up,” “movedown,” “clamp,” “unclamp,” “release,” “cut,” “stitch,” “seal,” “fire,”“squeeze,” etc. In some embodiments, the designated user may be requiredto say a password or an initiation word (e.g., “begin”) before issuingthe voice command for the surgical instrument 216. For example, thedesignated user may have to say “begin cut” for the processor 250 togenerate a command signal that instructs an endocutter in the surgicalsystem to cut a tissue. In such a case the command “cut,” by itself,will not be effective to generate a command signal from the processor tothe instrument to initiate cutting tissue. In some embodiments, afterdetecting the voice command from the designated user, the surgicalsystem may repeat the voice command, and ask the designated user forconfirmation (e.g., by saying “yes” or “no”). For example, after thedesignated user has said “begin cut,” the surgical system can repeat thevoice command and ask the designated user for permission to proceed. Ifthe designated user answers in an affirmative (e.g. “yes,” “proceed,”etc.) the processor 250 will generate a command signal instructing theendocutter in the surgical system to proceed with cutting the tissue. Insome embodiments, the voice commands can include the name of thesurgical instrument 216. For example, the designated user may say “Movegrasper right” to instruct a grasper to move to the right.

If the processor 250 does not recognize the voice captured by themicrophone (e.g., when the microphone captures the voice of the persons202 or 206), a command signal is not generated. In some embodiments,processor 250 may inform the personnel in the surgical environment 200that the captured voice could not be recognized (e.g., by anannouncement through a speaker in the surgical environment 200). If theprocessor 250 does not find a match between the voice command by thedesignated user and the voice commands in the voice command data fileassociated with the surgical instrument 216, a command signal is notgenerated. Additionally, the personnel in the surgical environment 200are notified that a command signal will not be generated.

In some embodiments, certain functions of the surgical instrument can becontrolled by a user who may not be the designated user. For example,the surgical system may be configured to allow a trainer and/orsupervisory surgeon to control certain predetermined operations of thesurgical instrument 216. For example, a trainer can stop or pause theoperation of the surgical instrument configured to be used based onvoice command of the designated user. The trainer can control thepredetermined operations of the surgical instrument 216 by sending acontrol signal to the processor 250 using an input device. Alternatelyor additionally, the surgical system can be configured to capture andverify the voice command of the trainer. For example, the trainer canhave a microphone and a position indicator which allow the surgicalsystem to capture and verify the voice command of the trainer in amanner similar to that of the designated user described above. Thesurgical system can be configured to operate in a “training mode” inwhich one or more trainers can control the predetermined operations ofthe surgical instrument 216. Once the training mode is turned off, theprivileges of the trainers can be rescinded.

The surgical environment 200 can include a filter (e.g., band-passfilter, low-pass filter, high-pass filter) that allows acousticfrequency within a certain range to be captured and/or stored. In oneembodiment, the filter can include an acoustic filter coupled with themicrophone configured to capture the designated user's voice.Additionally, or alternately, the voice sensors (e.g., 212, 214, etc.)can also include the acoustic filter. The acoustic filter can be abandpass filter configured to allow acoustic frequencies correspondingto the frequency of designated user's voice. As an example, thebandwidth and location of the band-pass filter can be related to thegender of the designated user (e.g., 85 Hz-180 Hz for an adult male, and165 Hz-255 Hz for an adult female). In another embodiment, the signal(e.g., voltage or current signal) generated by the microphone can befiltered by an electronic filter (e.g., analog or digital circuitfilters). The electronic filters can be programmable (e.g., thebandwidth, filter-type, etc. can be varied). The processor 250 can beconfigured (e.g., based on a request of a user) to vary the propertiesof the electronic filter. The electronic filters can be configured toblock all frequencies except for a desired band of acoustic frequencies.

FIG. 3 illustrates an example of the surgical instrument 216 (e.g., anendocutter). The surgical instrument 216 can include multiple voicesensors 302, 304, 306 configured to detect sound generated by a source310 in the surgical environment 200. The surgical instrument 216 cansend one or more signals to the processor 250 that include informationrelated to the detected sound, time of detection of sound by the variousvoice sensors, etc. For example, the voice sensors can detect voicecommands generated by the designated user 204. The surgical instrument216 is also configured to receive command signals from the processor 250and based on such command signals the instrument can carry out anoperation. The surgical instrument 216 can optionally include a switch318 and in some embodiments the switch is be activated for the surgicalinstrument 216 to carry out an operation. In some embodiments, theswitch 318 can be remoted actuated (e.g., wirelessly through WiFi,Bluetooth, etc.).

FIG. 4 illustrates an example of a headset 400 that can be worn by thedesignated user 204. The headset 400 can also include a camera 402 andmultiple microphones 404, 406. The microphones are configured to detecta voice signal generated by the designated user 204. The camera 402,which is optional, is configured to capture images of the mouth of thedesignated user 402. The microphones and the camera 402 are configuredto be in communication with the processor 250 (e.g., wirelessly) and toconvey information related to captured image and/or detected voicesignal to the processor 250. Additionally, information related to thetime of detection of the voice signal and image can be conveyed to theprocessor 250 as well. In one embodiment, the camera is configured tocapture the motion of the lips of the designated user during thegeneration of the voice signal. The processor can use an imagerecognition algorithm to decipher the words/phrase corresponding to thecaptured motion of the lips of the designated user. The processor 250can compare the deciphered words/phrases with the voice signal generatedby the designated user approximately at the same time that thecorresponding motion of lips is captured. If the result of the attemptedauthentication is an affirmative (i.e., the deciphered phrase and thevoice signal match), the processor 250 will generate a command signalcorresponding to the voice signal.

One or more aspects or features of the subject matter described hereincan be realized in digital electronic circuitry, integrated circuitry,specially designed application specific integrated circuits (ASICs),field programmable gate arrays (FPGAs) computer hardware, firmware,software, and/or combinations thereof. These various aspects or featurescan include implementation in one or more computer programs that areexecutable and/or interpretable on a programmable system including atleast one programmable processor, which can be special or generalpurpose, coupled to receive data and instructions from, and to transmitdata and instructions to, a storage system, at least one input device,and at least one output device. The programmable system or computersystem may include clients and servers. A client and server aregenerally remote from each other and typically interact through acommunication network. The relationship of client and server arises byvirtue of computer programs running on the respective computers andhaving a client-server relationship to each other.

The computer programs, which can also be referred to as programs,software, software applications, applications, components, or code,include machine instructions for a programmable processor, and can beimplemented in a high-level procedural language, an object-orientedprogramming language, a functional programming language, a logicalprogramming language, and/or in assembly/machine language. As usedherein, the term “machine-readable medium” refers to any computerprogram product, apparatus and/or device, such as for example magneticdiscs, optical disks, memory, and Programmable Logic Devices (PLDs),used to provide machine instructions and/or data to a programmableprocessor, including a machine-readable medium that receives machineinstructions as a machine-readable signal. The term “machine-readablesignal” refers to any signal used to provide machine instructions and/ordata to a programmable processor. The machine-readable medium can storesuch machine instructions non-transitorily, such as for example as woulda non-transient solid-state memory or a magnetic hard drive or anyequivalent storage medium. The machine-readable medium can alternativelyor additionally store such machine instructions in a transient manner,such as for example as would a processor cache or other random accessmemory associated with one or more physical processor cores.

To provide for interaction with a user, one or more aspects or featuresof the subject matter described herein can be implemented on a computerhaving a display device, such as for example a cathode ray tube (CRT) ora liquid crystal display (LCD) or a light emitting diode (LED) monitorfor displaying information to a user and a keyboard and a pointingdevice, e.g., a mouse, a trackball, etc., by which a user may provideinput to the computer. Other kinds of devices can be used to provide forinteraction with a user as well. For example, feedback provided to auser can be any form of sensory feedback, such as for example visualfeedback, auditory feedback, or tactile feedback; and input from a usermay be received in any form, including, but not limited to, acoustic,speech, or tactile input. Other possible input devices include, but arenot limited to, touch screens or other touch-sensitive devices such assingle or multi-point resistive or capacitive trackpads, voicerecognition hardware and software, optical scanners, optical pointers,digital image capture devices and associated interpretation software,and the like.

FIG. 5 illustrates an exemplary embodiment of a computer system 500. Asshown, the computer system 500 includes one or more processors 502 whichcan control the operation of the computer system 500. “Processors” arealso referred to herein as “controllers.” The processor(s) 502 caninclude any type of microprocessor or central processing unit (CPU),including programmable general-purpose or special-purposemicroprocessors and/or any one of a variety of proprietary orcommercially available single or multi-processor systems. The computersystem 500 can also include one or more memories 504, which can providetemporary storage for code to be executed by the processor(s) 502 or fordata acquired from one or more users, storage devices, and/or databases.The memory 504 can include read-only memory (ROM), flash memory, one ormore varieties of random access memory (RAM) (e.g., static RAM (SRAM),dynamic RAM (DRAM), or synchronous DRAM (SDRAM)), and/or a combinationof memory technologies.

The various elements of the computer system 500 can be coupled to a bussystem 512. The illustrated bus system 512 is an abstraction thatrepresents any one or more separate physical busses, communicationlines/interfaces, and/or multi-drop or point-to-point connections,connected by appropriate bridges, adapters, and/or controllers. Thecomputer system 500 can also include one or more network interface(s)506, one or more input/output (IO) interface(s) 508, and one or morestorage device(s) 510.

The network interface(s) 506 can enable the computer system 500 tocommunicate with remote devices, e.g., other computer systems, over anetwork, and can be, for non-limiting example, remote desktop connectioninterfaces, Ethernet adapters, and/or other local area network (LAN)adapters. The IO interface(s) 508 can include one or more interfacecomponents to connect the computer system 500 with other electronicequipment. For non-limiting example, the IO interface(s) 508 can includehigh speed data ports, such as universal serial bus (USB) ports, 1394ports, Wi-Fi, Bluetooth, etc. Additionally, the computer system 500 canbe accessible to a human user, and thus the IO interface(s) 508 caninclude displays, speakers, keyboards, pointing devices, and/or variousother video, audio, or alphanumeric interfaces. The storage device(s)510 can include any conventional medium for storing data in anon-volatile and/or non-transient manner. The storage device(s) 510 canthus hold data and/or instructions in a persistent state, i.e., thevalue(s) are retained despite interruption of power to the computersystem 500. The storage device(s) 510 can include one or more hard diskdrives, flash drives, USB drives, optical drives, various media cards,diskettes, compact discs, and/or any combination thereof and can bedirectly connected to the computer system 500 or remotely connectedthereto, such as over a network. In an exemplary embodiment, the storagedevice(s) can include a tangible or non-transitory computer readablemedium configured to store data, e.g., a hard disk drive, a flash drive,a USB drive, an optical drive, a media card, a diskette, a compact disc,etc.

The elements illustrated in FIG. 5 can be some or all of the elements ofa single physical machine. In addition, not all of the illustratedelements need to be located on or in the same physical machine.Exemplary computer systems include conventional desktop computers,workstations, minicomputers, laptop computers, tablet computers,personal digital assistants (PDAs), mobile phones, and the like.

The computer system 500 can include a web browser for retrieving webpages or other markup language streams, presenting those pages and/orstreams (visually, aurally, or otherwise), executing scripts, controlsand other code on those pages/streams, accepting user input with respectto those pages/streams (e.g., for purposes of completing input fields),issuing HyperText Transfer Protocol (HTTP) requests with respect tothose pages/streams or otherwise (e.g., for submitting to a serverinformation from the completed input fields), and so forth. The webpages or other markup language can be in HyperText Markup Language(HTML) or other conventional forms, including embedded Extensible MarkupLanguage (XML), scripts, controls, and so forth. The computer system 500can also include a web server for generating and/or delivering the webpages to client computer systems.

In an exemplary embodiment, the computer system 500 can be provided as asingle unit, e.g., as a single server, as a single tower, containedwithin a single housing, etc. The single unit can be modular such thatvarious aspects thereof can be swapped in and out as needed for, e.g.,upgrade, replacement, maintenance, etc., without interruptingfunctionality of any other aspects of the system. The single unit canthus also be scalable with the ability to be added to as additionalmodules and/or additional functionality of existing modules are desiredand/or improved upon.

A computer system can also include any of a variety of other softwareand/or hardware components, including by way of non-limiting example,operating systems and database management systems. Although an exemplarycomputer system is depicted and described herein, it will be appreciatedthat this is for sake of generality and convenience. In otherembodiments, the computer system may differ in architecture andoperation from that shown and described here.

Preferably, components of the invention described herein will beprocessed before use. First, a new or used instrument is obtained and ifnecessary cleaned. The instrument can then be sterilized. In onesterilization technique, the instrument is placed in a closed and sealedcontainer, such as a plastic or TYVEK bag. The container and instrumentare then placed in a field of radiation that can penetrate thecontainer, such as gamma radiation, x-rays, or high energy electrons.The radiation kills bacteria on the instrument and in the container. Thesterilized instrument can then be stored in the sterile container. Thesealed container keeps the instrument sterile until it is opened in themedical facility.

Typically, the device is sterilized. This can be done by any number ofways known to those skilled in the art including beta or gammaradiation, ethylene oxide, steam, and a liquid bath (e.g., cold soak).An exemplary embodiment of sterilizing a device including internalcircuitry is described in more detail in U.S. Pat. No. 8,114,345 filedFeb. 8, 2008 and entitled “System And Method Of Sterilizing AnImplantable Medical Device.” It is preferred that device, if implanted,is hermetically sealed. This can be done by any number of ways known tothose skilled in the art.

One skilled in the art will appreciate further features and advantagesof the invention based on the above-described embodiments. Accordingly,the invention is not to be limited by what has been particularly shownand described, except as indicated by the appended claims. Allpublications and references cited herein are expressly incorporatedherein by reference in their entirety.

What is claimed is:
 1. A surgical system, comprising: a plurality ofvoice sensors located in a surgical environment and configured to detectsound and generate a first plurality of signals; a position indicator,in proximity to a designated user, configured to indicate a firstposition of the designated user and generate a second signalrepresentative of the first position; and a processor configured to:receive the first plurality of signals and the second signal, determine,based on the first plurality of signals, a second position, compare thedetected sound with a registered voice command of the designated userstored in a memory to verify the designated user's credentials, and senda command signal to a surgical instrument to carry out an operationrelated to the voice command based on at least one of the verificationof the designated user's credentials, the first position, and the secondposition.
 2. The system of claim 1, wherein the detected sound is avoice command generated by the designated user.
 3. The system of claim2, wherein the voice command is detected by at least one voice sensor ina headset worn by the designated user.
 4. The system of claim 3, whereinthe headset includes a camera configured to detect a motion of lips ofthe designated user during the generation of the voice command and tosend a signal related to the detected motion of lips to the processor.5. The system of claim 4, wherein the processor is configured todetermine whether the detected motion of lips of the designated usermatches the generated voice commands and to send the command signal whenthe detected motion of lips of the designated user matches the generatedvoice commands.
 6. The system of claim 2, wherein the voice command isdetected by at least one voice sensor disposed on the surgicalinstrument.
 7. The system of claim 1, wherein the second position isdetermined based on one or more delays in detection of the sound by oneor more of the plurality of voice sensors.
 8. The system of claim 1,wherein the processor is configured to send the command signal when thedifference between the first position and the second position is lessthan a predetermined value.
 9. The system of claim 1, wherein theprocessor is configured to compare the detected sound with theregistered voice command using a voice recognition algorithm.
 10. Thesystem of claim 1, wherein the surgical instrument includes a switch andthe surgical instrument carries out the operation related to the voicecommand when the switch is actuated.
 11. The system of claim 1, whereinthe processor is configured to send the command signal based on at leasttwo of the verification of the designated user's credentials, the firstposition, and the second position.
 12. A robotic surgical system,comprising: a robotic arm; a tool assembly removably coupled to therobotic arm and comprising a shaft extending distally from a housing andan end effector coupled to a distal end of the shaft, the end effectorbeing configured to treat tissue; a plurality of voice sensors locatedin a surgical environment housing the robotic arm and configured todetect sound and generate a first plurality of signals; a positionindicator, in proximity to a designated user, configured to indicate afirst position of the designated user and generate a second signalrepresentative of the first position; and a processor configured to:receive the first plurality of signals and the second signal, determine,based on the first plurality of signals, a second position, compare thedetected sound with a registered voice command of the designated userstored in a memory to verify the designated user's credentials, and senda command signal to the tool assembly to carry out an operation relatedto the voice command based on at least one of the verification of thedesignated user's credentials, the first position, and the secondposition.
 13. The robotic surgical system of claim 12, wherein thesecond position is determined based on one or more delays in detectionof the sound by one or more of the plurality of voice sensors.
 14. Amethod, comprising: detecting, by a plurality of voice sensors, soundproduced in a surgical environment; indicating, by a position sensor, afirst position associated with a designated user; determining, based onthe detected sound, a second position; comparing the detected sound withregistered voice commands stored in a memory to verify the designateduser's credentials; and enabling a surgical instrument to carry out anoperation related to the voice command based on at least one of theverification of the user's credential, the first position, and thesecond position.
 15. The method of claim 14, further comprisingregistering the designated user's voice.
 16. The method of claim 15,wherein the registered voice is stored in memory.
 17. The method ofclaim 14, wherein the second position is determined based on one or moredelays in detection of the sound by one or more of the plurality ofvoice sensors.
 18. The method of claim 14, wherein the processor sendsthe command signal when the difference between the first position andthe second position is less than a predetermined value.
 19. The methodof claim 14, wherein the processor sends the command signal based on atleast two of the verification of the designated user's credentials, thefirst position, and the second position.